Papers with Knowledge transfer
Pretrained Bidirectional Distillation for Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on language knowledge transfer from pretrained models to neural machine translation models. |
| Approach: | They propose to use masked language pretraining to efficiently transfer bidirectional language knowledge to NMT models. |
| Outcome: | The proposed method can significantly improve machine translation performance and achieve competitive or even better results than previous methods. |
Language Branch Gated Multilingual Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation do not allow knowledge transfer across languages. |
| Approach: | They propose a language branch gated multilingual neural machine translation module that encourages knowledge transfer within the same language branch. |
| Outcome: | The proposed approach significantly improves translation quality on middle- and low-resource languages over previous methods. |
Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are mainly trained on English data and struggle with low-resource languages. |
| Approach: | They propose to add a new language to Llama to improve classification accuracy for Persian tasks by aligning representations through bilingual pretraining and instruction datasets. |
| Outcome: | The proposed model performs on generation and classification tasks with no adverse impact and sometimes even improvements on English tasks. |
Across Programming Language Silos: A Study on Cross-Lingual Retrieval-Augmented Code Generation (2026.findings-acl)
Copied to clipboard
Qiming Zhu, Jialun Cao, Xuanang Chen, Weili Zhang, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Shing-Chi Cheung
| Challenge: | Current research on large language models with retrieval-augmented code generation (RACG) has focused on single-language settings, leaving their cross-lingual effectiveness underexplored. |
| Approach: | They construct a dataset covering 13 PLs with nearly 14K instances to study cross-lingual code knowledge transfer in RACG. |
| Outcome: | The proposed model shows unequal cross-lingual knowledge transfer even with direct injection and shows limited reliance on natural language information embedded in code when equipped with a code-specific retriever. |